{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 使用 🤗 PEFT 加载adapters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[参数高效微调（PEFT）方法](https://huggingface.co/blog/peft)在微调过程中冻结预训练模型的参数，并在其顶部添加少量可训练参数（adapters）。adapters被训练以学习特定任务的信息。这种方法已被证明非常节省内存，同时具有较低的计算使用量，同时产生与完全微调模型相当的结果。\n",
    "\n",
    "使用PEFT训练的adapters通常比完整模型小一个数量级，使其方便共享、存储和加载。\n",
    "\n",
    "<div class=\"flex flex-col justify-center\">\n",
    "  <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/peft/PEFT-hub-screenshot.png\"/>\n",
    "  <figcaption class=\"text-center\">与完整尺寸的模型权重（约为700MB）相比，存储在Hub上的OPTForCausalLM模型的adapter权重仅为~6MB。</figcaption>\n",
    "</div>\n",
    "\n",
    "如果您对学习更多关于🤗 PEFT库感兴趣，请查看[文档](https://huggingface.co/docs/peft/index)。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 设置"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先安装 🤗 PEFT：\n",
    "\n",
    "```bash\n",
    "pip install peft\n",
    "```\n",
    "\n",
    "如果你想尝试全新的特性，你可能会有兴趣从源代码安装这个库：\n",
    "\n",
    "```bash\n",
    "pip install git+https://github.com/huggingface/peft.git\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 支持的 PEFT 模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Transformers原生支持一些PEFT方法，这意味着你可以加载本地存储或在Hub上的adapter权重，并使用几行代码轻松运行或训练它们。以下是受支持的方法：\n",
    "\n",
    "- [Low Rank Adapters](https://huggingface.co/docs/peft/conceptual_guides/lora)\n",
    "- [IA3](https://huggingface.co/docs/peft/conceptual_guides/ia3)\n",
    "- [AdaLoRA](https://huggingface.co/papers/2303.10512)\n",
    "\n",
    "如果你想使用其他PEFT方法，例如提示学习或提示微调，或者关于通用的 🤗 PEFT库，请参阅[文档](https://huggingface.co/docs/peft/index)。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 加载 PEFT adapter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "要从huggingface的Transformers库中加载并使用PEFTadapter模型，请确保Hub仓库或本地目录包含一个`adapter_config.json`文件和adapter权重，如上例所示。然后，您可以使用`AutoModelFor`类加载PEFT adapter模型。例如，要为因果语言建模加载一个PEFT adapter模型：\n",
    "\n",
    "1. 指定PEFT模型id\n",
    "2. 将其传递给`AutoModelForCausalLM`类"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers import AutoModelForCausalLM, AutoTokenizer\n",
    "\n",
    "peft_model_id = \"ybelkada/opt-350m-lora\"\n",
    "model = AutoModelForCausalLM.from_pretrained(peft_model_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<Tip>\n",
    "\n",
    "你可以使用`AutoModelFor`类或基础模型类（如`OPTForCausalLM`或`LlamaForCausalLM`）来加载一个PEFT adapter。\n",
    "\n",
    "\n",
    "</Tip>\n",
    "\n",
    "您也可以通过`load_adapter`方法来加载 PEFT adapter。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers import AutoModelForCausalLM, AutoTokenizer\n",
    "\n",
    "model_id = \"facebook/opt-350m\"\n",
    "peft_model_id = \"ybelkada/opt-350m-lora\"\n",
    "\n",
    "model = AutoModelForCausalLM.from_pretrained(model_id)\n",
    "model.load_adapter(peft_model_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 基于8bit或4bit进行加载"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`bitsandbytes`集成支持8bit和4bit精度数据类型，这对于加载大模型非常有用，因为它可以节省内存（请参阅`bitsandbytes`[指南](https://huggingface.co/docs/transformers/main/zh/./quantization#bitsandbytes-integration)以了解更多信息）。要有效地将模型分配到您的硬件，请在[from_pretrained()](https://huggingface.co/docs/transformers/main/zh/main_classes/model#transformers.PreTrainedModel.from_pretrained)中添加`load_in_8bit`或`load_in_4bit`参数，并将`device_map=\"auto\"`设置为："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig\n",
    "\n",
    "peft_model_id = \"ybelkada/opt-350m-lora\"\n",
    "model = AutoModelForCausalLM.from_pretrained(peft_model_id, quantization_config=BitsAndBytesConfig(load_in_8bit=True))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 添加新的adapter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "你可以使用`add_adapter`方法为一个已有adapter的模型添加一个新的adapter，只要新adapter的类型与当前adapter相同即可。例如，如果你有一个附加到模型上的LoRA adapter："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers import AutoModelForCausalLM, OPTForCausalLM, AutoTokenizer\n",
    "from peft import PeftConfig\n",
    "\n",
    "model_id = \"facebook/opt-350m\"\n",
    "model = AutoModelForCausalLM.from_pretrained(model_id)\n",
    "\n",
    "lora_config = LoraConfig(\n",
    "    target_modules=[\"q_proj\", \"k_proj\"],\n",
    "    init_lora_weights=False\n",
    ")\n",
    "\n",
    "model.add_adapter(lora_config, adapter_name=\"adapter_1\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "添加一个新的adapter："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# attach new adapter with same config\n",
    "model.add_adapter(lora_config, adapter_name=\"adapter_2\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在您可以使用`set_adapter`来设置要使用的adapter。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# use adapter_1\n",
    "model.set_adapter(\"adapter_1\")\n",
    "output = model.generate(**inputs)\n",
    "print(tokenizer.decode(output_disabled[0], skip_special_tokens=True))\n",
    "\n",
    "# use adapter_2\n",
    "model.set_adapter(\"adapter_2\")\n",
    "output_enabled = model.generate(**inputs)\n",
    "print(tokenizer.decode(output_enabled[0], skip_special_tokens=True))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 启用和禁用adapters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "一旦您将adapter添加到模型中，您可以启用或禁用adapter模块。要启用adapter模块："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers import AutoModelForCausalLM, OPTForCausalLM, AutoTokenizer\n",
    "from peft import PeftConfig\n",
    "\n",
    "model_id = \"facebook/opt-350m\"\n",
    "adapter_model_id = \"ybelkada/opt-350m-lora\"\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_id)\n",
    "text = \"Hello\"\n",
    "inputs = tokenizer(text, return_tensors=\"pt\")\n",
    "\n",
    "model = AutoModelForCausalLM.from_pretrained(model_id)\n",
    "peft_config = PeftConfig.from_pretrained(adapter_model_id)\n",
    "\n",
    "# to initiate with random weights\n",
    "peft_config.init_lora_weights = False\n",
    "\n",
    "model.add_adapter(peft_config)\n",
    "model.enable_adapters()\n",
    "output = model.generate(**inputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "要禁用adapter模块："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.disable_adapters()\n",
    "output = model.generate(**inputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 训练一个 PEFT adapter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PEFT适配器受`Trainer`类支持，因此您可以为您的特定用例训练适配器。它只需要添加几行代码即可。例如，要训练一个LoRA adapter：\n",
    "\n",
    "\n",
    "<Tip>\n",
    "\n",
    "如果你不熟悉如何使用`Trainer`微调模型，请查看[微调预训练模型](https://huggingface.co/docs/transformers/main/zh/training)教程。\n",
    "\n",
    "</Tip>\n",
    "\n",
    "1. 使用任务类型和超参数定义adapter配置（参见`LoraConfig`以了解超参数的详细信息）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from peft import LoraConfig\n",
    "\n",
    "peft_config = LoraConfig(\n",
    "    lora_alpha=16,\n",
    "    lora_dropout=0.1,\n",
    "    r=64,\n",
    "    bias=\"none\",\n",
    "    task_type=\"CAUSAL_LM\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. 将adapter添加到模型中。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add_adapter(peft_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3. 现在可以将模型传递给`Trainer`了！"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "trainer = Trainer(model=model, ...)\n",
    "trainer.train()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "要保存训练好的adapter并重新加载它："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.save_pretrained(save_dir)\n",
    "model = AutoModelForCausalLM.from_pretrained(save_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--\n",
    "TODO: (@younesbelkada @stevhliu)\n",
    "-   Link to PEFT docs for further details\n",
    "-   Trainer  \n",
    "-   8-bit / 4-bit examples ?\n",
    "-->"
   ]
  }
 ],
 "metadata": {},
 "nbformat": 4,
 "nbformat_minor": 4
}
